The operation of a conventional mechanical or optical pointing or input device such as a mechanical or optical computer mouse is well known in the art. By use of these devices, the user can select files, programs, or actions from lists, groups of icons, etc., and can “gesturally” move files, programs, etc. issue commands or map to specific actions, for example in drawing programs.
As examples, a mechanical computer mouse relies on one or more wheels and/or balls to track movement or displacement information relative to forward-backward and left-to-right movement of the computer mouse, for example by interrupting infrared beams of light directed at light sensors to create pulses representative of wheel or ball movement. Simple logic circuits interpret the relative timing of the pulses to indicate which direction the wheel(s) or ball(s) is moving, which is then converted by driver software into motion of a visual indicator such as a pointer, cursor, or cross-hair along X and Y axes of a computing device display screen.
An optical computer mouse replaces the mechanical mouse wheels or balls with one or more light sources such as light-emitting diodes (LEDs) or laser diodes to detect movement of the mouse relative to an underlying surface such as a mouse pad. The inertial/gyroscopic computer mouse uses a tuning fork or other accelerometer to detect rotary movement for every axis supported, most commonly using 2 degrees of rotational freedom and being insensitive to spatial translation. The user need only perform small wrist rotations to move a pointer or cursor on a display screen.
Typically, the above conventional types of computer mice do not provide the option of three-dimensional (3D) pointing or input, but rather control the motion of a pointer or cursor in a 2D graphical user interface. However, three-dimensional pointing and input have long been wanted features in human-computer interactions. Such 3D pointing/input allows a user to accomplish tasks that are not possible with a two-dimensional (2D) pointing device such as the conventional computer mice discussed above. For example, a 3D pointing device allows the user to perform tasks such as 3D sculpturing or drawing, “flying” in multi-dimensional space (for example, three-dimensional space defined by X, Y, and Z axes) such as during gaming, etc.
In order to accomplish 2D or 3D pointing, it is a requirement to be able to identify the location of a pointing device with respect to a reference point in 2D or 3D space. In the case of 2D pointing, this is relatively simple. For instance, a conventional optical or mechanical computer mouse can easily measure a distance traveled with respect to a previous location in both X and Y directions in any moment and, consequently, can report its current location in real time and display a cursor or other visible marker on a display screen of a computing device accordingly.
Three-dimensional pointing/input is more complex. There are two conventional ways to accomplish 3D pointing depending on whether the pointing device has a distance measuring component or not. If the pointing device has a distance measuring component, an imager such as an IR camera may be integrated into the pointing device to detect lights from an IR emitter of a console such as the console of a gaming device, and calculate spatial coordinates for the pointing device accordingly. The Wii® Remote marketed by Nintendo® falls within that category. A problem with this approach is that the pointing device spatial coordinates can only be calculated when its imager has a direct line of sight to a sensor bar associated with the gaming device console.
If the pointing device does not include a distance measuring component, a separate component is required to measure the distance between the pointing device and, for example, a gaming device or base station, or to identify the location of the pointing device with respect to the gaming device or base station. All gesture-based pointing device approaches, such the Kinect® device marketed by Microsoft®, belong to this latter category. In this case, the fingers or the hands of a user play the role of a pointing device and a special imaging device is required to identify the locations of the fingers or hands of the user. Three-dimensional mice such as 3Dconnexion/Logitech's® SpaceMouse® in the early 1990s and Kantek's® 3D RingMouse® in the late 1990s, also known as bats, flying mice or wands, also fall in this category. As an example, the RingMouse® was tracked by a base station through ultrasound. This approach has been found to be unable to provide sufficient resolution.
There have been prior art attempts to combine a pointing component and an imaging component into a single device. For example, a prior art computer mouse included a digital camera mounted into a housing of the mouse (see FIG. 1). This device also included a mode selection system allowing the device to transition between a 2D mouse function and a digital camera function. However, the prior art device cannot function as a 3D pointer.
To the author's knowledge, no pointing devices exist wherein a pointing/input device such as computer mouse configured for conventional 2D pointing can also be configured as a 3D pointing device in a 3D human-computer interaction system.